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Abstract — In previous work, we demonstrated how decoding 
of a non-binary linear code could be formulated as a linear- 
programming problem. In this paper, we study different poly- 
topes for use with linear-programming decoding, and show that 
for many classes of codes these polytopes yield a complexity ad- 
vantage for decoding. These representations lead to polynomial- 
time decoders for a wide variety of classical non-binary linear 
codes. 

I. Introduction 

In [1] and [2], the decoding of binary LDPC codes using 
linear-programming decoding was proposed, and the con- 
nections between linear-programming decoding and classical 
belief propagation decoding were established. In [3], the 
approach of [2] was extended to coded modulation, in par- 
ticular to codes over rings mapped to non-binary modulation 
signals. In both cases, the principal advantage of the linear- 
programming framework is its mathematical tractability [2], 
[3]. 

For the binary coding framework, alternative polytope rep- 
resentations were studied which gave a complexity advantage 
in certain scenarios [1], [2], [4], [5]. Analagous to the work 
°f UL [2], [4], [5] for binary codes, we define two polytope 
representations alternative to that proposed in [3] which offer a 
smaller number of variables and constraints for many classes 
of nonbinary codes. We compare these representations with 
the polytope in [3]. These representations are also shown to 
have equal error-correcting performance to the polytope in [3]. 

II. Linear-programming Decoding 

Consider codes over finite quasi-Frobenius rings (this in- 
cludes codes over finite fields, but may be more general). 
Denote by D\ such a ring with q elements, by its additive 
identity, and let = $H\{0}. Let C be a linear code of 
length n over £R with m x n parity-check matrix 7i. 

Denote the set of column indices and the set of row 
indices of H by X = {1, 2, • • • , n} and J = {1, 2, • • • , m}, 
respectively. The notation Hj will be used for the j-th row 
of TL. Denote by supp(c) the support of a vector c. For 
each j G J, let 2j = supp(Hj) and dj = |X/|, and let 



Given any c G 9t", parity check j G J is satisfied by c if 
and only if the following equality holds over 9\: 
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(1) 



For j G J, define the single parity check code Cj by 

Cj = {(bt) ieXj : "! 

Note that while the symbols of the codewords in C are indexed 
by I, the symbols of the codewords in Cj are indexed by X,. 
Observe that c G C if and only if all parity checks j G J are 
satisfied by c. 

Assume that the codeword c = (c\,C2, • • • ,c„) G C has 
been transmitted over a g-ary input memoryless channel, and 
a corrupted word y — (yi, ■ ■ ■ , y n ) G S" has been 
received. Here E denotes the set of channel output symbols. 
In addition, assume that all codewords are transmitted with 
equal probability. 

For vectors / G R' 9-1 '™, the notation 

/ = (A I /a I • • ■ I fn) , 
will be used, where 

VzGX, f { = (/,f Q) ) QeK - . 
We also define a function A : S — > (M U {ioo})^ 1 by 

A = (A (a) W , 
where, for each y G E, a G 

A^)=log(«' 

\p(y\<x), 

and p(y\c) denotes the channel output probability (density) 
conditioned on the channel input. Extend A to a map on E" 
by X(y) = (A( yi ) | X(y 2 ) \ ... | A(y„)). 

The LP decoder in [3] performs the following cost function 
minimization: 



(f,w) = arg min \(y)f 



(2) 



where the polytope Q is a relaxation of the convex hull of 
all points / G I'' -1 '", which correspond to codewords; this 



polytope is defined as the set of / e IR.' 9 together with 
the auxiliary variables 

Wj,b for j £ J,b e Cj , 
which satisfy the following constraints: 

Vj G J, Vb e Cj, w jjb >0, (3) 

Vj G J, E = 1 ' < 4 > 
bee, 

and 

Vj g J, Vi elj, Va g £R~, 

/< ) = Ebec i ,6 i =c« u; j,b • (5) 

The minimization of the objective function over Q forms 
the relaxed LP decoding problem. The number of variables and 
constraints for this LP are upper-bounded by n(q— 1) + mq d ~ 1 
and m(q 1 + d(q — 1) + 1) respectively. 

It is shown in [3] that if / is integral, the decoder out- 
put corresponds to the maximum-likelihood (ML) codeword. 
Otherwise, the decoder outputs an 'error' . 

III. New LP Description 

The results in this section are a generalization of the high- 
density polytope representation [2, Appendix II]. Recall that 
the ring Dt contains q—1 non-zero elements. Correspondingly, 
for vectors fc G N 9_1 , we adopt the notation 

Now, for any j G J, we define the mapping 

Kj : Cj — ► BJ 9-1 , 

b i ► Kj(b) 

defined by 

(Kj(b)) a = \{i G Zj : bi ■ Hj t i = a}\ 

for all a G lEH - . We may then characterize the image of Kj, 
which we denote by Tj, as 

% ■ = \ fc G N 9_1 : E a • fc Q = and E fc Q < • I , 

for each j G iZ, where, for any fc e N, a £ JR, 

f if fc = 

[ a + h a if fc > (fc terms in sum) 

The set % is equal to the set of all possible vectors Kj (b) for 
bGCj. 

Note that Kj is not a bijection, in general. We say that a 
local codeword b G Cj is fc-constrained over Cj if Kj(6) = fc. 

Next, for any index set V C Z, we introduce the following 
definitions. Let iV = |T|. We define the single-parity-check- 
code, over vectors indexed by T, by 




Also define a mapping Kr : Cr — ► N 9 1 by 
(«r(a)) a = |{jer : a* = a}| , 

and define, for fc G Tj, 

4*° = {aeC r : K r (a) = fc} . 

Below, we define a new polytope for decoding. Recall that 
V = {Vii 2/2) • • • j Vn) G S n stands for the received (corrupted) 
word. In the sequel, we make use of the following variables: 

• For all i G Z and all a G Dt _ , we have a variable /j . 
This variable is an indicator of the event = a. 

• For all j 6 J and fc G Zj, we have a variable Uj.k- 
Similarly to its counterpart in [2], this variable indicates 
the contribution to parity check j of fc-constrained local 
codewords over Cj. 

• For all j G J, % G Zj, fc G Zj, a G we have a 
variable #| °p fe . This variable indicates the portion of 
assigned to fc-constrained local codewords over Cj. 

Motivated by these variable definitions, for all j G J we 
impose the following set of constraints: 

Vielj,Va£fR-, /f 3 =$>S!fc- (7) 

fee?} 

E = 1 • < 8 > 

fcer 3 

Vfc G Zj,Va G 9T, 

E = *<* • °>> • (9) 

Vi G Zj , Vfc G Zj- , Va G 9T , zgfe > . (10) 
Vi G Zj,Vfc 6 7~, 

E E (id 

We note that the further constraints 

Vi G Z,Va G 9T, < fl a) < 1 , (12) 

VjeJ,VkeTj, Q<*3, h <l, (13) 

and 

Vj G J, Vi G Zj , Vfc G Tj , Va G 3T , zgfc ^ a ^ ' < 14 > 

follow from constraints (TTb-dTTb. We denote by £Y the polytope 
formed by constraints (TTb-dTTb. 

Let T = maxjgj- \Tj\. Then, upper bounds on the number 
of variables and constraints in this LP are given by n(q — 1) + 
m(d(q-l) + l)Tmdm(d(q-l) + l)+m((d+l)(q-l)+d)T, 
respectively. Since T < ( d+ ^ 1 ), the number of variables and 
constraints are 0(mq- d q ), which, for many families of codes, 
is significantly lower than the corresponding complexity for 
polytope Q. 
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For notational simplicity in proofs in this paper, it is 
convenient to define a new set of variables as follows: 



Vj e J,Wi e Z,-,Vfc e 2},Va e 
T («) = 



E 



y (/3) 



(15) 



/3GSR-, /3Hj,i=a 

Then constraints © and (fTTT i may be rewritten as 

Vj eJ.fee^Vaer, £ T \f k = k a ■ a hk , (16) 

Vje J, Vie T^VkeTj, < £ rg^ < ct,- fc . (17) 

Note that the variables r do not form part of the LP de- 
scription, and therefore do not contribute to its complexity. 
However these variables will provide a convenient notational 
shorthand for proving results in this paper. 

We will prove that optimizing the cost function (O over 
this new polytope is equivalent to optimizing over Q. First, 
we state the following proposition, which will be necessary to 
prove this result. 

Proposition 3.1: Let M e N and fc e Also let 

r C I, Assume that for each a G fR~, we have a set of 
nonnegative integers X ( Q ) — {xf^ : i G T} and that together 
these satisfy the constraints 

ier 

for all a G IK - , and 

E ^ ^ M 

for all i G T. 

Then, there exist nonnegative integers ^w a : aeC^} 



(18) 



(19) 



such that 

i) Wa = M 



2) For all a G 9t _ , i G T, 



E 



(20) 



(21) 



A sketch of the proof of this proposition will follow at the 
end of this section. We now prove the main result. 

Theorem 3.2: The set U = {/ : 3 cr, z s.t. (/, cr, z) G 
U} is equal to the set Q = {/ : 3 w s.t. (f,w) G Q}. 
Therefore, optimizing the linear cost function (01 over U is 
equivalent to optimizing over Q. 

Proof: 

1) Suppose, (/, w) G Q. For all j G J , fc G 7}, we define 

= E W 3> b ' 

bGCj, K.j(b)=fc 

and for all j G ^7, i £lj, fc G 7j, a G we define 

(a) \ ' 

bECj, K.j{b)—h : bi—a 



It is straightforward to check that constraints (TTOb 

and (TTTb are satisfied by these definitions. 

For every j G J , i el,, a G 9t~, we have by (0 



(a) _ 



2^ 



E 



E 



E 



(a) 



and thus constraint © is satisfied. 
Next, for every j G J, we have by (0| 



1 



= E E 

beCj fceTj bGCj,rej(b)=fc 



= E ' 

fc err- 
and thus constraint ^ is satisfied. 
Finally, for every j G J 1 fc G X,, a G 



E 



a; 



(/3) 



E E 



W,,b 



E E ^ 

beCj, Kj(b)=fc ieXj, biHj,i=a 
beCj , re,, (b)=fc 

Thus, constraint (O is also satisfied. This completes the 
proof of the first part of the theorem. 
2) Now assume (/, cr, z) is a vertex of the polytope U, 
and so all variables are rational, as are the variables r. 
Next, fix some j G J, fc G X, , and consider the sets 



(a) 



-.(«*) 
T »J,fc 



i G I, 



for a G 5H _ . By constraint ( fTTI i. for each a G 9t~, all 
the values in the set Xq are rational numbers between 
and 1. Let [i be the lowest common denominator of 
all the numbers in all the sets Xq U \ a G %K~~ . Let 

f t (q) 1 
Af(«) = J n ■ -hZ:* : ielj} , 
{ a iM J 

for each a G The sets X^ consist of integers 
between and fi. By constraint ( fTBT ), we must have that 
for every a G the sum of the elements in X^ is 
equal to k a /j,. By constraint ( TTTb . we have 

V'i.j.fc , 



aeSR- 



for all i e Ij. 

We now apply the result of Proposition 13 . 1 1 with T =Ij, 
M = jU and with the sets A^") defined as above (here 
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a E Cp fc ' ) } according 



N = dj). Set the variables {w D 
to Proposition 13.11 

Next, for fc G 7J-, we show how to define the variables 
{w' b : b E Cj, Kj(b) = k}. Initially, we set w' b = 
for all b E Cj, Kj(b) = k. Observe that the values 
fj, ■ z\jk/o'j,k are non-negative integers for every i £ 
1, j ej, keTj, /3 e 91-. 

(k) 

For every a E C r , we define w a words 
b^- 1 ', ir 1 ', ■ ■ ■ ,b' tUa - ) E Cj. Assume some ordering on 
the elements (3 E satisfying /3Hji = a,-, namely 
/3i,/?2,--- ,/?£ for some positive integer £ Q . For i E 



2,-, b 



l,2,---,w a ) is defined as follows: 

(ft 



6^ ; is equal to /3i for the first fj, ■ z\ ?y 1 o-j.k words 



',(1) 1,(2) 



is equal to 0% for the next 
A 4 ' z i^jk/ a j,k worc ls, and so on. For every b E Cj we 
define 

{t E {1,2, •• .,tfl } : b {l) =b} 
Finally, for every b £ Cj, Kj(b) — k, we define 

— -t«b ■ 

fj, 



W 3,b = 

Using Proposition 13. II 



E 



«' 



Oj.fc 



E 



/i ■ 



<Ji,k 



aeC^ 

and so all b (2) , • • ■ ,6 ( <" a) (for all a E C r fc) ) are 
well-defined. It is also straightforward to see that b^ E 
Cj for £ = 1, 2, • • ■ ,w a . Next, we check that the newly- 
defined Wj.b satisfy ©-(HJ for every j E J, b E Cj. 
It is easy to see that Wj.b > 0; therefore (01 holds. By 
Proposition 13. 1 1 we obtain 



Oj,fc 



E 

beCj, Kj(b)=k 



for all j E J,k and 



(«) ^ 



E 

beCj, K 3 (6)=fe, biHj,i=a 



for all j E J, ielj, fc E 7j, a E 9t — . Let /3Hj 
By the definition of Wj_b it follows that 

E W 3' b 
beCj, K(b)=k, bi=/3 



a. 



E 



w 



\i.b 



Z 



T i,j,k beCj, K(6)=fe, biH jt i=a 

where the first equality is due to the definition of the 

words b {l \ e=l,2,--- ,w a . 

By constraint dHJ we have, for all j E J , 

= E a i- k 

= E E w J> b = E w ^ b > 

fcGTj bGCj, Kj(b)=k beCj 



1 



thus satisfying ©. 

Finally, by constraint (O we obtain, for all j E J , i E 

E 91-, 



(ft _ Jft 



E 

keTj 



E w J- b 



E 

beCj, Kj(6)=fe, 6j=/3 

thus satisfying ©. 
Sketch of the Proof of Proposition \3.1\ 

In this proof, we use a network flow approach (see [6] for 
background material). 

The proof will be by induction on M. We set w a = for all 
a E Cp . We show that there exists a vector a = (a,i) ier E 
such that 



(i) For every i e T and a E 9t 



(ii) If for some i E T, ^ a e«- 
some a E D^ - . 



^ (a) > . 



» 



= M, then a, = a for 



Then, we 'update' the values of x[ a ^'s and M as follows 



For every i € T and a E 91 with a, = a we set x 



(a) 



» 



M - 



1. We also set ui Q 



a;>~' — 1. In addition, we set M 
w a + 1. 

It is easy to see that the 'updated' values of x y ""s and M 
satisfy 



» 



< M for all i E T. 



for all a E and £ Q 

Therefore, the inductive step can be applied with respect to 
these new values. The induction ends when the value of M is 
equal to zero. 

It is straightforward to see that when the induction termi- 
nates, d20b and (f2T~b hold with respect to the original values 
of the x[ a) and M. 

Existence of a that satisfies (i): We construct a flow network 
G = (V, E) as follows: V = {s, t} U Ui U U 2 , where Ui = 
and U 2 = T. Also set 



E = {{s,a)} ae x- U {(i,t)} ier U {(a,i)} m 



>o 



We define an integral capacity function c : E — > MU {+00} 
as follows: 

{k a if e = (s, a), a E 
1 if e = (i, i), i £ r . (22) 

+00 if e = (a, i), a E 9^ _ , i e T 

Next, apply the Ford-Fulkerson algorithm on the network 
(G(E,V),c) to produce a maximal flow f ma x. Since all the 
values of c(e) are integral for all e E E, so the values of 
fmax(e) must all be integral for every e E E (see [6]). 

It can be shown that the minimum cut in this graph has 
capacity c min = J2 ae m- k a- 
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The flow f max in G has a value of E Q g«R- ^a- Observe that 
fmax((a,0) 6 {0, 1} for all a G m~ and i G T. Then, for all 
i G r, we define 

_ | a if f m ax ((cm)) = 1 for some a G Ui 
1 1 otherwise 

For this selection of a — (ai, 02, • • ■ , ajv), we nave a G Cp fc ' 
and ai = a only if a;^ > 0. 

Existence of a that satisfies ( i) and ( ii) simultaneously: We 
start with the following definition. 

Definition 3.1: The vertex i G U 2 is called a critical 
vertex, if £ qG k- x\ a) = M. 

In order to have (TT9T > satisfied after the next inductive step, we 
have to decrease the value of EaeJK- x i by (exactly) 1 for 
every critical vertex. This is equivalent to having f max ((i, t)) = 
1. 

We aim to show that there exists a flow f * of the same value, 
which has f*((i,t)) = 1 for every critical vertex i. Suppose 
that there is no such flow. Then, consider the maximum flow 
f, which has f'((i, t)) = 1 for the maximal possible number of 
the critical vertices i G UV We assume that there is a critical 
vertex in, G U2, which has f'((io,t)) = 0. It is possible to 
show that the flow f can be modified towards the flow f " of 
the same value, such that for f" the number of critical vertices 
j e U2 having f"((i,t)) = 1 is strictly larger than for f. 

It follows that there exists an integral flow f* in (G(V, E), c) 
of value J2 a em- sucn ^ or evel T critical vertex i G U2, 
f*((i,t)) = 1. We define 

_ ( a if f*((a, i)) = 1 for some a G Ui 
* 1 otherwise 

and a = (aj)igr. For this selection of a, we have a G C^ 
and the properties (i) and (ii) are satisfied. □ 

IV. Cascaded Polytope Representation 

In this section we show that the "cascaded polytope" 
representation described in [4] and [5] can be extended to non- 
binary codes in a straightforward manner. Below, we elaborate 
on the details. 

For j G J, consider the j-th row Hj of the parity-check 
matrix H over Dl, and recall that 

Ci = {(&i)iei, : £>-W,-,i = 0}. 

Assume that Tj = *2 S • * " >idA an d denote Cj = 
{1, 2, ■ • • , dj — 3}. We introduce new variables x' = [xi)ieCj 
and denote x = (x^jeJ- 

We define a new linear code Cj of length 2dj — 3 by 
(dj — 2) x (2dj — 3) parity-check matrix associated with the 
following set of parity-check equations over £H: 

1) KKjA + K'riiM + Xi = • (23) 

2) For every t = 1, 2, • • • , dj - 4, 

- + K+J-Lj^ + xi+i = . (24) 

3) - <- 3 + Ki-Jij,^-. + = ■ (25) 



We also define a linear code of length n + J2j e j(dj — 3) 
defined by (J2 j&J (dj - 2)) x (n + E^K" ~ 3 )) P^ity- 
check matrix T associated with all the sets of parity-check 
equations d23l-(l25l (for all j G J). 

Theorem 4.1: The vector (6j)igx. G -Dt 3 is a codeword 
of Cj if and only if there exists some vector % J G 9i dj ~ 3 such 

that ((fii) ieXj I X J ) G Cf. 

We denote by S the polytope corresponding to the LP 
relaxation problem O)-© for the code C^' with the parity- 
check matrix T . Let (6, x) be a word in C^- x \ where b G C. 
It is natural to represent points in S as ((f,h),z), where 

/ = Cfi^iEX, ag!R- and = (hj" )j£>7, i££ 3 -, aeSH- are 

vectors of indicators corresponding to the entries bi (i G I) 
in b and (j ^ Ji i & £j) in x, respectively. 

Theorem 4.2: The set S = {/ : 3 h, z s.t. ((/, h), z) G 
5} is equal to the set Q — {/ : 3 w s.t. (f,w) G Q}, and 
therefore, optimizing the linear cost function over S is 
equivalent to optimizing it over Q. 

It follows from Theorem 14.21 that the polytope S equiv- 
alent^ describes the code C. This description has at most 
n+m-(d— 3) variables and m-(d— 2) parity-check equations. 
However, the number of variables participating in every parity- 
check equation is at most 3. Therefore, the total number of 
variables and of equations in the respective LP problem will 
be bounded from above by 

(n + m(d - 3))(g - 1) + m(d - 2) • q 2 

and m(d - 2)(q 2 + 3q ~ 2) . 

The polytope representation in this section, when used with 
the LP problem in [3], leads to a polynomial-time decoder for 
a wide variety of classical non-binary codes. Its performance 
under LP decoding is yet to be studied. 
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